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ABSTRACT 


Hindcast  analysis  of  OTH  radar  data  differs  from  purely  predictive  (or  forecast) 
analysis  in  that  it  is  made  "after  the  fact"  of  radar  measurement,  permitting  the  removal  of  a 
substantial  part  of  the  random  variability  inherent  in  forecasting  radar  performance.  In  this 
paper,  hindcast  analysis  is  used  to  verify  the  performance  of  the  MITRE  OTH  radar 
performance  model,  HFRAD.  Data  was  collected  during  November  and  December  of  1990 
w'ith  the  AN,TPS-1 18  East  Coast  Radar  System  (ECRS),  and  during  January  and  February'  of 
1991  with  the  West  Coast  Radar  System  (WCRS).  This  data  was  analyzed  and  compared  to 
hindcast  estimates  of  performance  using  a  new  version  (V507g)  of  HFRAD  that  incorporates 
a  new  and  improved,  clutter  model.  Performance  was  analyzed  in  terms  of  target  signal-to- 
noise  ratio  (SNR),  both  for  the  aggregate  data  set  and  for  the  data  as  a  function  of  azimuth 
(because  of  its  location  near  the  auroral  oval,  the  ECRS  is  known  to  exhibit  a  strong 
dependence  of  performance  on  azimuth).  In  addition,  these  internal  model  components  were 
analyzed  in  the  snme  way:  subclutter  visibility  (SCV),  surface  clutter  peak,  background 
noise  (i.e.,  noiss.  ..easured  with  transmitter  operating),  noise  measured  with  transmitter  off 
(ambient  noise),  and  the  ratio  of  background  noise  to  ambient  noise.  As  anticipated,  the 
comparison  is  generally  in  good  agreement,  although  notable  discrepancies  still  exist, 
particularly  near  the  auroral  oval.  Suggestions  for  further  improvements  to  HFRAD  are 
presented. 
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The  performance  of  Over-the  Horizon  (OTH)  radar  is  strongly  dependent  on 
ionospheric  propagation  and,  by  extension,  on  ionospheric  structure  as  well.  The  ionosphere 
is  a  highly  dynamic  propagation  medium  that  exhibits  both  deterministic  and  random 
components  of  variability.  In  predicting  the  performance  of  OTH  radars,  it  is  first  necessary 
to  predict  ionospheric  structure  and  then  compute  the  propagation  modes  and  losses 
experienced  by  the  radar  energy.  Ionospheric  climatological  models  exist  for  the 
deterministic  component,  but  the  random  component  must  be  modeled  in  statistical  terms. 
This  creates  difficulties  in  testing  other  components  of  radar  performance  models,  since 
several  of  these  have  strong  frequency  dependences;  a  prime  example  is  radar  clutter 
magnitude,  which  has  an  inverse  cube  frequency  dependence. 

One  way  to  avoid  that  difficulty  is  to  compare  OTH  radar  data  with  a  model  in  which 
the  random  component  has  been  removed  to  the  extent  possible  by  measuring  the  ionospheric 
parameters  prevailing  during  the  measurement  and  using  the  actual  radar  frequency,  rather 
than  the  frequency  that  would  be  predicted  by  the  propagation  model.  This  technique  is 
known  as  "hindcasting,"  as  opposed  to  the  purely  predictive  technique.  This  paper  describes 
a  hindcast  analysis  designed  to  test  the  accuracy  of  the  MITRE  OTH  radar  performance 
prediction  program,  HFRAD,  using  data  collected  with  the  U.S.  Air  Force  AN/FPS-1 18 
radar. 
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Hindcast  consists  of  running  HFRAD  after  the  fact  (e.g.,  after  a  radar  measurement 
has  been  made)  with  the  actual  values  of  radar  frequency  and  waveforms,  sunspot  number 
(SSN),  magnetic  activity  (Kp),  and  ionospheric  layer  parameters.  Using  the  actual 
ionospheric  parameters  allows  the  propagation  mode  used  by  HFRAD  to  agree  with  that 
measured  during  the  data  collection  period.  However,  the  removal  of  uncertainty  in  the 
operation  frequency,  system  sensitivity  parameters,  and  propagation  mode  does  not 
completely  eliminate  all  the  model  variances;  for  example,  as  will  be  shown  later,  there  is 
still  a  substantial  uncertainty  in  the  propagation  loss. 

HFRAD  contains  several  major  model  components  that  are  used  in  computing  the 
signal-to-noise  ratio  (SNR)  for  a  given  radar  cross  section  (RCS).  Figure  1  illustrates  the 
major  components  that  are  involved  in  the  OTH  radar  range  equation.  The  major  model 
components  include  the  following:  (1)  radar  equipment  model  (coherent  integration  time,  t; 
signal  processing  loss,  Lgp;  transmit  antenna  gain,  Gt;  receive  antenna  directivity.  Dr; 

transmit  power,  Pt);  (2)  propagation  model  (propagation  mode  to  target,  1F2,  IFl,  etc.;  slant 
range,  R;  maximum  usable  frequency  (MUF);  focusing  factor,  F;  ionospheric  layer 
parameters,  foF2,  foFl,  foE,  foEs;  sporadic-E  loss,  LesJ  ionospheric  D-region  non-deviative 
absorption,  Lp;  auroral  absorption,  LaUR)J  (3)  noise  model  (atmospheric,  man-made,  and 
system  noise,  N);  (4)  clutter  model  (only  outer  Doppler  clutter,  C);  and  (5)  radar  cross  section 
(target,  at)  and  surface  reflectivity  (Gq). 

Some  of  the  above  GTH  radar  model  components  can  be  directly  measured,  others 
must  be  inferred  from  a  combination  of  measurements,  and  some  are  not  available;  for 
example,  in  the  radar  range  equation 


(4pfR^Lr(N-^C) 


(1) 


where  Lj  —  +  Lu  + 
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Figure  1.  HFRAD  OTH  Radar  Range  Equation 


we  are  not  able  to  measure  the  individual  loss  terms,  except  perhaps  the  signal  processing 
loss,  if  a  calibration  signal  is  available.  The  SNR  of  a  target  or  the  amplitude  of  peak  surface 
return  consists  of  a  complicated  interrelationship  of  radar,  propagation,  and  ionospheric 
parameters.  Hence,  there  are  limitations  in  our  ability  to  validate  the  various  HFRAD  model 
components. 

From  November  1990  through  February  1991,  MITRE  was  extensively  involved  in 
East  Coast  Radar  System  (ECRS)  and  West  Coast  Radar  System  (WCRSj  data  collection, 
processing,  and  analysis  for  the  purpose  of  quantifying  the  operational  performance  of  the 
ECRS  and  WCRS  for  single  targets.  The  ECRS  and  WCRS  database  consists  of  radar,  flight 
plan  (assumed  target  position  and  velocity),  and  ionospheric  measurements.  Both  the  ECRS 
and  WCRS  measurements  are  associa.ed  either  with  aircraft  that  have  been  tracked  and 
correlated,  or  with  the  mid-path  location  of  the  flight  plan  within  the  illuminated  coverage 
area.  In  this  paper  we  only  present  ECRS  HFRAD/measurement  results.  All  measured 
parameters  are  defined  with  respect  to  a  coherent  signal  at  the  output  of  a  single  antenna 
element  or  at  the  input  of  a  single  receiver  in  units  of  dBm.  In  other  words,  the  signal 
processing  and  receive  array  gain  have  been  backed  out  of  the  measurement.  Since  the  noise 
and  clutter  signals  are  assumed  to  be  incoherent  signals,  the  units  are  given  in  dBs  (or  scaled 
dB);  the  scaling  factor,  which  is  given  by  signal  processing  and  receive  array  gains,  needs  to 
be  calculated  to  convert  dBs  to  dBm.  For  the  AN/FPS-1 15  radar,  the  scaling  factor  is  given 
approximately  by 


Scaling  Factor  {Sp) »  CIT 4-  ARC  -  L^p 
-^3  +  19-8  =  14  dB 


where  CIT  is  the  radar  waveform  coherent  integration  time,  ARC  is  the  array  receiver  gain, 
and  Lsp  is  the  signal  processing  loss  for  a  coherent  signal. 
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The  measured  parameters  are  defined  by  the  following: 

SCV  (dB)  =  Subclutter  visibility.  The  ratio  of  the  un-normalized  surface  return 
clutter  peak  to  the  background  noise  or  transmitter-on  noise  level. 

CLP  (dBm)  =  Surface  clutter  peak. 

TXN  (dBs)  =  Transmitter-off  noise.  TXN  is  an  incoherent  signal,  but  is  scaled  as  a 
coherent  signal  with  respect  to  the  output  of  a  single  antenna 
element. 

BGN  (dBs)  =  Transmitter-on  noise.  The  sum  of  transmitter-off  noise  (TXN)  plus 
spread-Doppler  clutter  (ICL),  derived  from  the  SCV  and  CLP 
measurements. 

BNR  (dB)  =  Background-to-noise  ratio.  Defined  by  the  following  expression 

BNR^BGN  -TXN  (dB) 

SNR  (dB)  =  Median  signal-to-noise  ratio  (SNR).  Noise  in  SNR  is  referred  to  as 
transmitter-on  noise.  The  median  is  computed  from  the  first  15  hits 
after  track  establishment. 

Measured  radar  waveform  parameter  definitions  include  the  following: 

Frequency  (MHz)  =  Measured  at  time  of  TET  or  TMFP  if  no  correlated  track 

PWBW  (kHz)  =  Primary  waveform  bandwidth 

PCIT  (sec)  =  Primary  waveform  coherent  integration  time 

PWRF  (Hz)  =  Primary  waveform  repetition  frequency 

SWBW  (kHz)  =  Secondary  waveform  bandwidth 

Serr  (sec)  =  Secondary  waveform  coherent  integration  time 

SWRF  (Hz)  =  Secondary  waveform  repetition  frequency 
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As  noted  earlier,  hindcasting  is  expensive  in  terms  of  required  analysis  resources.  For 
this  reason,  we  selected  only  about  20  percent  of  the  available  ECRS  data  to  be  hindcast. 

This  corresponds  to  hindcasting  nine-day  and  seven-night  ECRS  missions  (each  mission 
corresponds  to  about  a  four-hour  time  interval).  A  hindcast  day  mission  was  defined  to  take 
place  in  the  interval  1300-1900  Universal  Time  (UT),  while  a  hindcast  night  mission  took 
place  from  0100-04(X)  UT.  Only  segment  1  (16.5  degrees  to  76.5  degrees  from  true  north) 
flight  plan  data  was  selected  for  the  ECRS. 

During  the  data  collection  period  of  November  through  February',  the  HFRAD- 
predicted  radar  frequency,  ionospheric  layer  parameters,  Kp,  and  SSN  were  very  close  to  the 
monthly  median  values  predicted  by  HFRAD.  The  geomagnetic  conditions  for  that  period 
were  quiet  to  unsettled.  Figure  2  shows  a  plot  of  the  average  daily  and  monthly  Kp  the  filled 
diamonds  show  the  ECRS  hindcast  days  during  November  and  December.  The  monthly 
average  Kp  is  for  November  and  December  were  about  2*  and  1+  respectively.  The  daily 
average  Kp  for  the  hindcast  missions  varied  from  a  maximum  of  about  3*  to  a  low  of  about 
Kp  =  1  with  a  mean  of  about  Kp  =  2’.  The  ECRS  hindcast  missions  represent  a  fairly  broad 
sampling  of  day  and  night  missions  that  includes  periods  of  both  good  and  poor  radar 
performance. 

Several  program  modifications  were  added  to  HFRAD  to  enable  hindcasting 
capabilities  such  as  user-specified  radar  frequency;  user-specified  ionospheric  layer 
parameters  (foF2,  foFl,  foE,  foEs);  user-specified  foEs  standard  deviation  (used  in  the 
sporadic-E  signal  loss  integration  computation);  user-specified  vertical  incidence  sounder 
location;  and  user-  specified  WRF  (waveform  repetition  frequency). 

The  generation  of  environmental  files  includes  the  following  information: 

1 .  Radar  operation  frequency  (MHz) 

2.  Ionospheric  layer  parameters  (foF2,  foFl,  foE,  foEs,  in  MHz) 

3.  Vertical  incidence  sounder  location  (geographic  coordinates) 

4.  foEs  standard  deviation  (MHz) 

5.  Geomagnetic  activity  (Kp) 
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ge  Daily  Kp 


6.  Solar  activity  (SSN) 

7.  Time-of-day  Oocal  or  Universal,  in  hours) 

8.  Month 

9.  Azimuth  (degrees  from  True  North) 

10.  Range  start  (km) 

11.  Range  sampling  interval  (km) 

12.  Number  of  range  intervals 

Another  important  HFRAD  hindcast  input  file  is  the  radar  file  that  includes  the 
following  information: 

1 .  Transmitter  power 

3.  Target  radar  cross  section  (large  aircraft) 

4.  Target  aspect  angle 

5.  Target  velocity 

6.  Radar  waveform  (CIT  and  bandwidth;  WRF) 

The  measured,  predictive,  and  measured-minus-predictive  cumulative  distributions  of 
daytime  and  nighttime  frequencies  are  shown  in  figure  3.  The  distribution  curves  tend  to  fall 
along  several  straight  lines  occurring  with  different  break  points  in  frequency.  The  daytime 
measured  radar  frequency  distribution  appears  to  be  bimodal  with  a  median  daytime 
frequency  of  about  24.5  MHz  and  a  standard  deviation  (based  on  the  84.9  percent  point)  of 
about  1.5  MHz.  The  predicted  frequency  distribution  shows  a  bimodal  shape  similar  to  that 
of  the  measurement  distribution  plot,  with  a  break  point  at  28  MHz  corresponding  to  the 
radar’s  upper  frequency  limit.  The  meas-pred  frequency  distribution  exhibits  a  complex 
bimodal  shape,  and  it  appears  that  HFRAD  over-predicted  the  operating  frequency  by  about 
3  MHz.  This  may  be  related  to  the  lact  of  available  clear  operating  frequencies  in  the  range 
of  26  to  28  MHz.  The  apparent  bimodal  shape  of  the  distributions  is,  of  course,  strongly 
influenced  by  the  truncation  at  the  upper  end  of  the  frequency  range.  The  nighttime 
frequency  distributions  have  a  much  less  noticeable  break  point,  presumably  because 
spectrum  availability  is  more  uniform  and  no  equipment  limitation  exists,  as  is  the  case  in 
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Figure  3.  Cumulative  Distributions  of  Daytime  and  Nighttime  Frequencies 


daytime.  At  night,  the  measured  and  predicted  median  frequencies  were  closer  than  in  the 
daytime,  with  HFRAD  predicting  about  1  MHz  higher  than  the  measured  median  of  9  MHz. 

Figures  4  and  5  show  plots  of  daytime  and  nighttime  ECRS  measurements  and 
HFRAD  predictions  (hindcast)  of  the  sample  size  (number  of  flight  plans),  frequency,  SCV, 
SNR,  BNR,  clutter  peak,  background  noise,  and  TX-OFF  as  a  function  of  azimuth.  The 
HFRAD  hindcast  results  were  obtained  by  using  the  current  baseline  version  of  HFRAD. 

The  sample  size  number  represents  the  total  number  of  measurements  (i.e.,  flight  plans)  for  a 
given  segment.  We  used  flight  plans  instead  of  "cases,"  since  the  number  of  flight  plans  per 
case  varies  with  azimuth;  the  number  of  measurement  samples  is  used  in  determining 
statistical  significance.  Note  that  the  sample  size  increases  from  low  azimuths  to  high 
azimuths,  and  it  is  larger  in  the  daytime  than  at  night,  which  is  consistent  with  the  flight  plan 
data. 


The  measurements  and  HFRAD  hindcast  prediction  comparisons  of  frequency 
suggest  that  the  radars  were  operated  nearly  optimally  (during  both  day  and  night  in  the  case 
of  the  ECPS).  As  noted  earlier,  it  appears,  however,  that  some  restriction  existed  on  the 
upper  frequency  limit  of  the  ECRS,  perhaps  related  to  spectral  occupancy,  which  would  have 
degraded  daytime  performance  by  ~  1-2  dB.  The  azimuthal  comparisons  of  the  radar 
performance  parameters  (SCV,  SNR,  BNR,  CLP,  clutter,  noise,  etc.)  show  a  strong  azimuthal 
(or  latitudinal)  variation  in  the  performance  parameters.  The  largest  differences  appear  to  be 
associated  with  the  HFRAD  clutter  model.  They  include  the  following: 

1.  HFRAD  under-predicting  clutter  by  about  1  to  4  dB  in  sectors  2,  5,  6,  7,  and  8  in 
the  daytime  for  ECRS 

2.  HFRAD  over-predicting  clutter  by  about  4  to  10  dB  in  segments  2,  3,  and  4  and 
by  about  1  to  4  dB  in  segments  5  through  8  in  the  nighttime  for  ECRS 

These  azimuthal  clutter  comparison  results  strongly  suggest  that  the  HFRAD  clutter  model 
still  needs  further  improvement. 
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Figure  4.  ECRS  Daytime  Hindcast  HFRADAleasurement  Comparisons 
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Figure  5.  ECRS  Nighttime  Hindcast  IIFRAD/Measurement  Comparisons 


The  noise  model/measurement  comparisons  show  fairly  good  agreement  during  the 
nighttime  for  the  ECRS,  but  over-predict  TX-OFF  noise  in  sectors  2,  3,  and  4  in  the  daytime. 
HFRAD  under-predicts  the  daytime  surface  clutter  peak  (CLP)  by  about  3  dB  in  the  more 
southerly  sectors  (i.e.,  5-8)  of  ECRS  segment  1,  but  it  over-predicts  CLP  by  about  10  dB  in 
sector  2  of  ECRS  segment  1  during  the  daytime.  This  over-prediction  of  CLP  may  be 
attributed  either  to  anomalous  ionospheric  absorption  (i.e.,  winter  anomaly  or  auroral 
absorption)  or  variability  in  surface  reflectivity  (i.e.,  sea  ice  instead  of  ocean  water).  At 
nighttime,  the  clutter  CLP  is  over-predicted  by  about  5  dB  in  sectors  4  through  8. 

An  interesting  comparison  is  the  measurement-hindcast  difference  result.  Figures  6 
and  7  show  the  daytime  and  nighttime  measurement-hindcast  comparisons,  respectively,  as  a 
function  of  azimuth.  Also  shown  is  the  STDEV  associated  with  the  measurement-hindcast 
comparison.  As  may  be  anticipated,  the  measurement-hindcast  comparison  indicates  closer 
agreement  between  measurement  and  HFRAD  hindcast  at  azimuths  greater  than  40  degrees 
(lower  latimdes),  and  poor  agreement  at  the  higher  azimuths  (less  than  40  degrees).  Note  that 
the  STDEV  also  appears  larger  at  the  more  northerly  azimuths  (or  higher  latitudes). 

The  TX-OFF  comparisons  in  figures  6  and  7  show  that  HFRAD  tends  to  over-predict 
TX-OFF  noise  by  about  3  dB  at  night  and  with  the  greatest  STDEV  at  more  northerly 
azimuths.  The  daytime  comparisons  show  HFRAD  over-predicting  TX-OFF  by  as  much  as 
7  dB  at  the  higher  latitudes  (in  ECRS  sectors  2,  3,  and  4),  but  show  fairly  good  agreement  for 
sectors  5  through  8.  This  suggests  an  azimuthal  noise  dependence,  possibly  related  to  man¬ 
made  sources,  that  is  not  built  into  the  International  Radio  Consultative  Committee  (CCIR) 
noise  model  (which  is  based  on  measurements  with  an  antenna  having  an  isotropic  radiation 
pattern).  It  is  also  possible  that  auroral  absorjltion,  which  is  primarily  a  daytime 
phenomenon,  is  influencing  the  measured  noise  values. 

In  summary,  the  hindcasting  technique  permits  separate  model  components  to  be 
isolated,  to  some  extent,  and  is  therefore  a  valuable  aid  in  improving  the  accuracy  of  the 
overall  performance  model.  By  removing  the  uncertainty  inherent  in  predicting  the 
ionospheric  structure  prevailing  during  a  radar  measurement,  as  well  as  that  associated  with 
the  selection  of  the  radar  operating  frequency,  hindcasting  has  been  used  in  this  paper  to 
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Figure  7.  Mean  of  <Meas-l  lindcasD*  for  ECRS  Nighttime 


fcKus  attention  on  the  clutter,  noise,  and  propagation  loss  components  of  HFRAD.  It  is 
recommended  that  further  work  be  done  in  the  areas  of  modeling  Doppler-spread  clutter, 
propagation  loss,  focusing,  and  noise  modeling.  In  addition,  it  was  recognized  that  the  basic 
propagation  model  embodied  in  HFRAD  should  be  improved  to  allow  for  inclusion  of  effects 
of  horizontal  gradients  in  the  ionosphere. 
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J.  C.  Tang 

L.  J.  Teig 

D080 

N.  M.  Tomljanovich 
L.  D.  Tromp 

R.  Wm.  Bush 

D.  J.  Violette 

R.  M.  Davis 

M.  M.  Weiner 

C.  H.  Gager 

A.  J.  Goldberg 

D090 

G.  L.  Guttrich 

J.  D.  R.  Kramer,  Jr. 

E.  D.  Ostroff 

P.  A.  Bello 
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DISTRIBUTION  LIST  (CONCL'D) 


D095 

J.  N.  Freedman 

G140 

D.  R.  Uffelman 

J103 

D.  H.  Durkin 

W090 

T.  J.  Elkins  (10) 
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